IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Assistant Commissioner for Patents Atty Dkt: 1767-43 £ ^ 

Washington, D.C. 20231 ^ 

*d for filing is the patent application of: Date: March 18, 1999 ^sg 

■-^^Bijor: 1 TAKAHASHI n ^ 

\^g§: MOVEMENT VECTOR GENERATING 
^g^= • APPARATUS AND METHOD, AND IMAGE 
ENCODING APPARATUS AND METHOD 
mfe r^ludinq attachments as noted below: 
El Declaration, g| Abstract 

58 pages of specification and claims (including numbered claims), and 

8 sheets of accompanying drawing/s. 

g| Record & return the attached assignment to the undersigned. 

[g] Priority is hereby claimed under 35 USC 119 based on the following foreign applications: 

Application Number Country Day/Month/Year Filed 

1 0-92673 Japan 20 March 1 998 

, respectively. 

Certified copy(ies) of foreign application(s): [x] attached: □ filed on 



in U.S. Application Serial No. filed on 

or in PCT Application No. filed 



Please amend the specification by inserting before the first line -This is a of PCT application , filed 

Priority is hereby claimed under 35 USC 120/365 based on the following prior PCT applications designating the U.S.: 
H Application Number Country Day/Month/Year Filed 

IS] This application is based on the following prior provisional application(s): 
iijl Application No. Filing Date 

p respectively, and priority is hereby claimed therefrom. 

|£P Please amend the specification by inserting before the first line: - This application claims the benefit of U.S. 
l2 Provisional Application No. , filed 

j jp Verified Statement attached establishing "small entity" status (Rules 9 & 27) 

: |b The Examiner's attention is directed to the prior art cited in the parent application by applicant and/or Examiner for 
^ the reasons stated therein. 

Preliminary amendment to claims (attached hereto), to be entered before calculation of the fee below. 
{J' Also attached: 

FILING FEE IS BASED ON CLAIMS AS FILED LESS ANY HEREWITH CANCELED 

Basic Filing Fee $ 760.00 

Total effective claims 14 - 20 (at least 20) = 0 x$ 18.00 $ o.OO 

Independent claims 4 -3 (at least 3) = 1 x$ 78.00 $ 78.00 

If any proper multiple dependent claims now added for first time, add $260.00 (ignore improper) $ 0.00 

SUBTOTAL $ 838.00 

If "small entity," then enter half (1/2) of subtotal and subtract -$( 0.00) 

SECOND SUBTOTAL $ 838.00 

Assignment Recording Fee ($40.00) $ 40.00 

TOTAL FEE ENCLOSED $ 878.00 

Any future submission requiring an extension of time is hereby stated to include a petition for such time extension. 
The Commissioner is hereby authorized to charge any deficiency in the fee(s) filed, or asserted to be filed, or which should have been 
filed herewith (or with any paper hereafter filed in this application by this firm) to our Account No. 14-1 140. A duplicate copy of this 
sheet is attached. 

1 100 North Glebe Road, 8th Floor NIXON & VANDERHJE P.C. 

Arlington, Virginia 22201-4714 

Telephone; (703)816-4000 By Atty: Arthur 

Facsimile: (703) 816-4100 

ARC:pdc Signature: 

330193 




OurRef.: 1767-43 
P01-3240/TE 



U.S. PATENT APPLICATION 



Inventor(s): Tsutomu Takahashi 



Invention: MOVEMENT VECTOR GENERATING APPARATUS AND METHOD, AND 

IMAGE ENCODING APPARATUS AND METHOD 



NIXON & VANDERHYE P.C. 

ATTORNEYS AT LAW 
1100 NORTH GLEBE ROAD 
8 th FLOOR 
ARLINGTON, VIRGINIA 22201-4714 
(703) 816-4000 
Facsimile (703) 816-4100 



SPECIFICATION 



MOVEMENT VECTOR GENERATING APPARATUS 
AND METHOD, AND 
IMAGE ENCODING APPARATUS AND METHOD 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a movement vector 
generating apparatus and an image encoding apparatus, for 
generating a movement vector corresponding to a movement 
compensation when the movement compensation involving an 
inter-frame prediction represented by a so-called MPEG 
(Moving Picture Experts Group) method is executed to 
compress and encode image information. 

2. Description of the Related Art 

A so-called movement compensating process is used in 
an image encoding process, such as the above-mentioned 
MPEG method and the like. 

In this movement compensating process, at first, an 
image to be encoded is divided into pixel blocks each including 
a predetermined number of pixels which is set in advance (e.g., 
divided into macro blocks in the case of the MPEG method). 
Then, an absolute value of a difference between each pixel 
within each pixel block and the corresponding pixel within 
any one frame located forward or backward on a time axis is 
calculated. Then, the sum is obtained by summing up the 
absolute values in all pixels within the pixel block. Then, a 



spatial position of the image, in which the above-mentioned 
sum of the absolute values becomes minimum is determined 
(i.e., the spatial position of the image which is located the 
closest to the image within the pixel block and belongs to any 
one frame located forward or backward on the time axis). 

Then, the relationship between the pixel block and the 
closest image thereto is defined as a movement vector. Then, 
this movement vector is encoded as information representing 
the image within any one frame located forward or backward 
on the time axis. Accordingly, the image information can be 
encoded with compressing a sufficient amount of the 
information to be actually encoded. 

In the above mentioned movement compensating 
process, when the closest image to the pixel block to be 
encoded is searched within any one frame located forward or 
backward on the time axis, a search range and a search 
accuracy within the frame are common for all the pixel blocks 
within the image information to be encoded. 

This search of the movement vector needs a vast 
amount of calculation. 

Therefore, if the searching process is simplified so as 
to reduce the calculation amount in the case of using the 
uniform search range and search accuracy for all the pixel 
blocks, the exact movement vector cannot be determined, 
which results in a deterioration of the image quality after the 
decoding operation, which is a problem. 
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On the other hand, although a uniform enlargement of 
the search range is effective in order to improve the image 
quality, with respect to an image having the heavy movement 
as a whole image of a single frame (for example, an image 
photographed when a camera photographing the image is 
moving (panning)), the calculation amount is further 
increased, which is also a problem. 

As a countermeasure for this, the uniform 
enlargement of the search range for a slender (little) 
movement causes the calculation amount to be increased. 
That is, a proper vector calculation cannot be performed, 
depending on the characteristic of the image information. 
This results in the problem of the deterioration of the image 
quality or the unnecessary increase of the calculation amount. 

SUMMARY OF THE INVENTION 
The present invention is proposed in view of the above 
mentioned problems. It is therefore an object of the present 
invention to provide a vector generating apparatus, a vector 
generating method, an image encoding apparatus having such 
a vector generating apparatus, and an image encoding method 
including such a vector generating method, which can 
properly generate a movement vector in response to the 
characteristics of an image to be encoded without 
deteriorating the quality of the image. 

The above object of the present invention can be 
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achieved by a movement vector generating apparatus for 
generating a movement vector for a movement compensation 
by means of an inter-frame prediction, when encoding a preset 
image information including an image of a plurality of frames 
by using the movement compensation. The movement vector 
generating apparatus is provided with: a plurality of 
generating devices each for generating the movement vector 
corresponding to a search range and a search accuracy 
between one frame and another frame, for each pixel block e.g., 
each macro block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, the generating devices respectively using search 
ranges different from each other and search accuracies 
different from each other; and a selecting device for selecting 
one of movement vectors generated by the generating devices, 
in accordance with characteristics of the image in said each 
pixel block, and then outputting the selected movement vector 
corresponding to said each pixel block. 

According to the movement vector generating 
apparatus of the present invention, the movement vector 
corresponding to the search range and the search accuracy 
between one frame and another frame are generated for each 
pixel block by each of the generating devices. At this time, 
the generating devices respectively using the search ranges 
different from each other and the search accuracies different 
from each other. Then, one of movement vectors generated 



4 



by the generating devices is selected by the selecting device, 
in accordance with characteristics of the image in each pixel 
block. Then, the selected movement vector corresponding to 
each pixel block is outputted from the selecting device. 

Accordingly, since the movement vector is selected in 
accordance with the characteristics of the image in each pixel 
block which is to be encoded by using the movement vectors 
generated with the search ranges and the search accuracies 
different from each other for each pixel block, it is possible to 
generate the movement vector with the search range and the 
search accuracy appropriate for the image in each pixel block. 
Therefore, by using the movement vector generated by the 
present invention when encoding the image information, it is 
possible to encode the image information at a higher quality 
as compared with the case that the movement vector is 
generated with the fixed search range and search accuracy. 

In one aspect of the movement vector generating 
apparatus of the present invention, the generating devices 
include: a first generator for generating a first movement 
vector, with a preset first range as the search range; and a 
second generator for generating a second movement vector at 
the search accuracy lower than that of the first movement 
vector, with a preset second range wider than the first range 
as the search range. 

According to this aspect, in the generating device, the 
first movement vector is generated with the first range as the 
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search range by the first generator. On the other hand, the 
second movement vector is generated by the second generator 
at the search accuracy lower than that of the first movement 
vector with the second range wider than the first range as the 
search range. 

Accordingly, it is possible to generate the movement 
vector with the search range and the search accuracy 
appropriate for the image in each pixel block. 

In another aspect of the movement vector generating 
apparatus of the present invention, the selecting device 
outputs the second movement vector as the selected movement 
vector if a length of the second movement vector is a length 
beyond the search range in the first generating device, and 
outputs the first movement vector as the selected movement 
vector if the length of the second movement vector is a length 
belonging to the search range in the first generating device. 

According to this aspect, if the length of the second 
movement vector is a length beyond the search range in the 
first generating device, the second movement vector is 
outputted as the selected movement vector by the selecting 
device. Alternatively, if the length of the second movement 
vector is a length belonging to the search range in the first 
generating device, the first movement vector is outputted as 
the selected movement vector by the selecting device. 

Accordingly, if the movement of the image is fine or 
minute in one pixel block, the movement vector can be 



generated at the high accuracy. On the other hand, if the 
movement of the image is large in one pixel block, the vector 
can be generated with the large search range. 

In another aspect of the movement vector generating 
apparatus of the present invention, the selecting device is 
provided with: a first adding device for adding together 
absolute values of differences between respective one of the 
pixels in the pixel block and its corresponding pixel in the 
frame targeted by the movement compensation, as for all of 
the pixels in the pixel block, in the first generating device, to 
generate a first absolute value sum; a second adding device 
for adding together absolute values of differences between 
respective one of the pixels in the pixel block and its 
corresponding pixel in the frame targeted by the movement 
compensation, as for all of the pixels in the pixel block, in the 
second generating device, to generate a second absolute value 
sum; and a standardizing device for standardizing the 
generated first and second absolute value sums, respectively, 
the selecting device comparing the standardized first absolute 
value sum with the standardized second absolute value sum, 
and outputting the second movement vector as the selected 
movement vector if the standardized first absolute value sum 
is greater than the standardized second absolute value sum. 

According to this aspect, in the selecting device, as for 
all of the pixels in the pixel block in the first generating 
device, the absolute values of differences between respective 
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one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the first adding device, so that the first 
absolute value sum is generated. On the other hand, as for 
all of the pixels in the pixel block in the second generating 
device, the absolute values of differences between respective 
one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the second adding device, so that the second 
absolute value sum is generated. Then, the generated first 
and second absolute value sums are respectively standardized 
by the standardizing device. Then, the standardized first 
absolute value sum is compared with the standardized second 
absolute value sum by the selecting device. As a result of 
the comparison, if the standardized first absolute value sum 
is greater than the standardized second absolute value sum, 
the second movement vector is outputted as the selected 
movement vector from the selecting device. 

Accordingly, it is possible to generate the movement 
vector such that the absolute value sum is minimized in the 
wider search range i.e., the movement vector indicates the 
closer image. 

In another aspect of the movement vector generating 
apparatus of the present invention, the selecting device is 
provided with: a first adding device for adding together 
absolute values of differences between respective one of the 
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pixels in the pixel block and its corresponding pixel in the 
frame targeted by the movement compensation, as for all of 
the pixels in the pixel block, in the first generating device, to 
generate a first absolute value sum; a second adding device 
for adding together absolute values of differences between 
respective one of the pixels in the pixel block and its 
corresponding pixel in the frame targeted by the movement 
compensation, as for all of the pixels in the pixel block, in the 
second generating device, to generate a second absolute value 
sum; and a standardizing device for standardizing the 
generated first and second absolute value sums, respectively, 
the selecting device comparing the standardized first absolute 
value sum with the standardized second absolute value sum, 
outputting the first movement vector as the selected 
movement vector if a difference between the standardized 
first absolute value sum and the standardized second absolute 
value sum is not greater than a predetermined threshold 
which is set in advance to detect a difference between the first 
movement vector and the second movement vector at a high 
accuracy, and outputting the second movement vector as the 
selected movement vector if the difference between the 
standardized first absolute value sum and the standardized 
second absolute value sum is greater than the predetermined 
threshold. 

According to this aspect, in the selecting device, as for 
all of the pixels in the pixel block in the first generating 
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device, the absolute values of differences between respective 
one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the first adding device, so that the first 
absolute value sum is generated. On the other hand, as for 
all of the pixels in the pixel block in the second generating 
device, the absolute values of differences between respective 
one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the second adding device, so that the second 
absolute value sum is generated. Then, the generated first 
and second absolute value sums are respectively standardized 
by the standardizing device. Then, the standardized first 
absolute value sum is compared with the standardized second 
absolute value sum by the selecting device. As a result of 
the comparison, if the difference between the standardized 
first absolute value sum and the standardized second absolute 
value sum is not greater than the threshold, the first 
movement vector is outputted as the selected movement vector 
from the selecting device. Alternatively, if the difference 
between the standardized first absolute value sum and the 
standardized second absolute value sum is greater than the 
threshold, the second movement vector is outputted as the 
selected movement vector from the selecting device. 

Accordingly, it is possible to generate the movement 
vector by predominantly selecting one of the first and second 
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movement vectors which has the higher accuracy and generate, 
even if the difference between the standardized first absolute 
value sum and the standardized second absolute value sum is 
small. 

In another aspect of the movement vector generating 
apparatus of the present invention, the selecting device 
outputs one of the first and second movement vectors which is 
closer to the selected movement vector corresponding to 
another pixel block located adjacent to one pixel block as the 
selected movement vector corresponding to said one pixel 
block from which the first movement vector and the second 
movement vector are generated. 

According to this aspect, one of the first and second 
movement vectors which is closer to the selected movement 
vector corresponding to another pixel block located adjacent 
to one pixel block is outputted by the selecting device as the 
selected movement vector corresponding to one pixel block. 

Accordingly, it is possible to generate the movement 
vector in consideration with the mutual relationship between 
the pixel blocks by using the fact that the approximate 
movement vectors are generated between the macro blocks 
adjacent to each other in many cases. 

The above object of the present invention can be also 
achieved by an image encoding apparatus provided with (a) 
the above described movement vector generating apparatus of 
the present invention, (b) a compensating device for 
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performing the movement compensation on the basis of the 
selected movement vector outputted from the selecting device, 
to output a compensation signal, and (c) an encoding device 
for encoding the image information on the basis of the 
compensation signal. 

According to the image encoding apparatus of the 
present invention, the selected movement vector is outputted 
from the above described movement vector generating 
apparatus of the present invention. Then, on the basis of 
this selected movement vector, the movement compensation is 
performed by compensating device, and the compensation 
signal is outputted therefrom. Then, on the basis of this 
compensation signal, the image information is encoded by the 
encoding device. 

Accordingly, since the movement compensation and 
the image encoding process are performed by use of the 
movement vector generated with the appropriate search range 
and search accuracy in correspondence with the image in each 
pixel block, it is possible to encode the image information with 
a higher accuracy as compared with the case that the 
movement vector is generated with the fixed search range and 
search accuracy. 

The above object of the present invention can be also 
achieved by a movement vector generating method of 
generating a movement vector for a movement compensation 
by means of an inter-frame prediction, when encoding a preset 
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image information including an image of a plurality of frames 
by using the movement compensation. The movement vector 
generating method is provided with: a plurality of generating 
processes each of generating the movement vector 
corresponding to a search range and a search accuracy 
between one frame and another frame, for each pixel block e.g., 
each macro block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, the generating processes respectively using search 
ranges different from each other and search accuracies 
different from each other; and a selecting process of selecting 
one of movement vectors generated by the generating 
processes, in accordance with characteristics of the image in 
said each pixel block, and then outputting the selected 
movement vector corresponding to said each pixel block. 

According to the movement vector generating method 
of the present invention, the movement vector corresponding 
to the search range and the search accuracy between one 
frame and another frame are generated for each pixel block by 
each of the generating process. At this time, the generating 
process respectively using the search ranges different from 
each other and the search accuracies different from each other. 
Then, one of movement vectors generated by the generating 
process is selected by the selecting process, in accordance 
with characteristics of the image in each pixel block. Then, 
the selected movement vector corresponding to each pixel 
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block is outputted from the selecting process. 

Accordingly, since the movement vector is selected in 
accordance with the characteristics of the image in each pixel 
block which is to be encoded by using the movement vectors 
generated with the search ranges and the search accuracies 
different from each other for each pixel block, it is possible to 
generate the movement vector with the search range and the 
search accuracy appropriate for the image in each pixel block. 
Therefore, by using the movement vector generated by the 
present invention when encoding the image information, it is 
possible to encode the image information at a higher quality 
as compared with the case that the movement vector is 
generated with the fixed search range and search accuracy. 

In one aspect of the movement vector generating 
method of the present invention, the generating processes 
include: a first generating process of generating a first 
movement vector, with a preset first range as the search 
range; and a second generating process of generating a second 
movement vector at the search accuracy lower than that of the 
first movement vector, with a preset second range wider than 
the first range as the search range. 

According to this aspect, in the generating process, 
the first movement vector is generated with the first range as 
the search range by the first generating process. On the 
other hand, the second movement vector is generated by the 
second generating process at the search accuracy lower than 
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that of the first movement vector with the second range wider 
than the first range as the search range. 

Accordingly, it is possible to generate the movement 
vector with the search range and the search accuracy 
appropriate for the image in each pixel block. 

In another aspect of the movement vector generating 
method of the present invention, the selecting process outputs 
the second movement vector as the selected movement vector 
if a length of the second movement vector is a length beyond 
the search range in the first generating process, and outputs 
the first movement vector as the selected movement vector if 
the length of the second movement vector is a length 
belonging to the search range in the first generating process. 

According to this aspect, if the length of the second 
movement vector is a length beyond the search range in the 
first generating process, the second movement vector is 
outputted as the selected movement vector by the selecting 
process. Alternatively, if the length of the second movement 
vector is a length belonging to the search range in the first 
generating process, the first movement vector is outputted as 
the selected movement vector by the selecting process. 

Accordingly, if the movement of the image is fine or 
minute in one pixel block, the movement vector can be 
generated at the high accuracy. On the other hand, if the 
movement of the image is large in one pixel block, the vector 
can be generated with the large search range. 
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In another aspect of the movement vector generating 
method of the present invention, the selecting process is 
provided with: a first adding process of adding together 
absolute values of differences between respective one of the 
pixels in the pixel block and its corresponding pixel in the 
frame targeted by the movement compensation, as for all of 
the pixels in the pixel block, in the first generating process, to 
generate a first absolute value sum; a second adding process 
of adding together absolute values of differences between 
respective one of the pixels in the pixel block and its 
corresponding pixel in the frame targeted by the movement 
compensation, as for all of the pixels in the pixel block, in the 
second generating process, to generate a second absolute 
value sum; and a standardizing process of standardizing the 
generated first and second absolute value sums, respectively, 
the selecting process comparing the standardized first 
absolute value sum with the standardized second absolute 
value sum, and outputting the second movement vector as the 
selected movement vector if the standardized first absolute 
value sum is greater than the standardized second absolute 
value sum. 

According to this aspect, in the selecting process, as 
for all of the pixels in the pixel block in the first generating 
process, the absolute values of differences between respective 
one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
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added together by the first adding process, so that the first 
absolute value sum is generated. On the other hand, as for 
all of the pixels in the pixel block in the second generating 
process, the absolute values of differences between respective 
one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the second adding process, so that the 
second absolute value sum is generated. Then, the 
generated first and second absolute value sums are 
respectively standardized by the standardizing process. 
Then, the standardized first absolute value sum is compared 
with the standardized second absolute value sum by the 
selecting process. As a result of the comparison, if the 
standardized first absolute value sum is greater than the 
standardized second absolute value sum, the second 
movement vector is outputted as the selected movement vector 
from the selecting process. 

Accordingly, it is possible to generate the movement 
vector such that the absolute value sum is minimized in the 
wider search range i.e., the movement vector indicates the 
closer image. 

In another aspect of the movement vector generating 
method of the present invention, the selecting process is 
provided with: a first adding process of adding together 
absolute values of differences between respective one of the 
pixels in the pixel block and its corresponding pixel in the 
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frame targeted by the movement compensation, as for all of 
the pixels in the pixel block, in the first generating process, to 
generate a first absolute value sum; a second adding process 
of adding together absolute values of differences between 
respective one of the pixels in the pixel block and its 
corresponding pixel in the frame targeted by the movement 
compensation, as for all of the pixels in the pixel block, in the 
second generating process, to generate a second absolute 
value sum; and a standardizing process of standardizing the 
generated first and second absolute value sums, respectively, 
the selecting process comparing the standardized first 
absolute value sum with the standardized second absolute 
value sum, outputting the first movement vector as the 
selected movement vector if a difference between the 
standardized first absolute value sum and the standardized 
second absolute value sum is not greater than a 
predetermined threshold which is set in advance to detect a 
difference between the first movement vector and the second 
movement vector at a high accuracy, and outputting the 
second movement vector as the selected movement vector if 
the difference between the standardized first absolute value 
sum and the standardized second absolute value sum is 
greater than the predetermined threshold. 

According to this aspect, in the selecting process, as 
for all of the pixels in the pixel block in the first generating 
process, the absolute values of differences between respective 
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one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the first adding process, so that the first 
absolute value sum is generated. On the other hand, as for 
all of the pixels in the pixel block in the second generating 
process, the absolute values of differences between respective 
one of the pixels in the pixel block and its corresponding pixel 
in the frame targeted by the movement compensation are 
added together by the second adding process, so that the 
second absolute value sum is generated. Then, the 
generated first and second absolute value sums are 
respectively standardized by the standardizing process. 
Then, the standardized first absolute value sum is compared 
with the standardized second absolute value sum by the 
selecting process. As a result of the comparison, if the 
difference between the standardized first absolute value sum 
and the standardized second absolute value sum is not greater 
than the threshold, the first movement vector is outputted as 
the selected movement vector from the selecting process. 
Alternatively, if the difference between the standardized first 
absolute value sum and the standardized second absolute 
value sum is greater than the threshold, the second movement 
vector is outputted as the selected movement vector from the 
selecting process. 

Accordingly, it is possible to generate the movement 
vector by predominantly selecting one of the first and second 
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movement vectors which has the higher accuracy and generate, 
even if the difference between the standardized first absolute 
value sum and the standardized second absolute value sum is 
small. 

In another aspect of the movement vector generating 
method of the present invention, the selecting process outputs 
one of the first and second movement vectors which is closer 
to the selected movement vector corresponding to another 
pixel block located adjacent to one pixel block as the selected 
movement vector corresponding to said one pixel block from 
which the first movement vector and the second movement 
vector are generated. 

According to this aspect, one of the first and second 
movement vectors which is closer to the selected movement 
vector corresponding to another pixel block located adjacent 
to one pixel block is outputted by the selecting process as the 
selected movement vector corresponding to one pixel block. 

Accordingly, it is possible to generate the movement 
vector in consideration with the mutual relationship between 
the pixel blocks by using the fact that the approximate 
movement vectors are generated between the macro blocks 
adjacent to each other in many cases. 

The above object of the present invention can be also 
achieved by an image encoding method provided with (a) the 
above described movement vector generating method of the 
present invention, (b) a compensating process of performing 
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the movement compensation on the basis of the selected 
movement vector outputted from the selecting process, to 
output a compensation signal, and (c) an encoding process of 
encoding the image information on the basis of the 
compensation signal. 

According to the image encoding method of the present 
invention, the selected movement vector is outputted from the 
above described movement vector generating method of the 
present invention. Then, on the basis of this selected 
movement vector, the movement compensation is performed 
by compensating process, and the compensation signal is 
outputted therefrom. Then, on the basis of this 
compensation signal, the image information is encoded by the 
encoding process. 

Accordingly, since the movement compensation and 
the image encoding process are performed by use of the 
movement vector generated with the appropriate search range 
and search accuracy in correspondence with the image in each 
pixel block, it is possible to encode the image information with 
a higher accuracy as compared with the case that the 
movement vector is generated with the fixed search range and 
search accuracy. 

The nature, utility, and further features of this 
invention will be more clearly apparent from the following 
detailed description with respect to preferred embodiments of 
the invention when read in conjunction with the 
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accompanying drawings briefly described below. 



BRIEF DESCRIPTION OF THE DRAWINGS 
FIG.l is a diagram showing the principle of a 

movement compensation; 

FIG. 2 is a block diagram showing a schematic 

configuration of an image encoding apparatus as a first 

embodiment of the present invention; 

FIG.3A is a block diagram showing a detailed 

structure of a movement detecting unit in the first 

embodiment; 

FIG.3B is a block diagram showing a detailed 
structure of a vector generating apparatus in the first 
embodiment; 

FIG.4A is a diagram showing a case in which a 
movement vector generated by a hierarchy search belongs to a 
search range of an entire search, in a selection of a movement 
vector in the first embodiment; 

FIG.4B is a view showing a case in which the 
movement vector generated by the hierarchy search does not 
belong to the search range of the entire search, in the 
selection of the movement vector in the first embodiment; 

FIG. 5 is a block diagram showing a schematic 
configuration of a movement detecting unit in a second 
embodiment of the present invention; 

FIG.6A is a block diagram showing a schematic 
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configuration of a movement detecting unit in a third 
embodiment; 

FIG.6B is a diagram showing the operation thereof; 

FIG, 7 is a block diagram showing a schematic 
configuration of a movement detecting unit in a fourth 
embodiment of the present invention; 

FIG. 8 is a block diagram showing a schematic 
configuration of an image encoding apparatus as a modified 
embodiment of the present invention; 

FIG. 9 is a TABLE (a) indicating a bit rate with 
respect to the entire search, the hierarchy search and the 
adaptation switch respectively, for an image of "succer"; and 

FIG. 10 is a TABLE (b) indicating a bit rate with 
respect to the entire search, the hierarchy search and the 
adaptation switch respectively, for an image of "horse race". 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Preferred embodiments of the present invention will 
be described below with reference to the drawings. The 
respective embodiments described below are embodiments in 
which the present invention is applied to a movement 
compensating process in an image encoding apparatus which 
compresses and encodes image information digitized for each 
pixel constituting each frame by using the above-mentioned 
MPEG method. 

( I ) Principle of Movement Compensating Process 
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At first, the principle of the movement compensating 
process according to the present invention is exemplified and 
explained with reference to FIG.l, before actually explaining 
the embodiments. 

At first, as shown in FIG.l, it is assumed that there 
are two frames (in the case of FIG.l, a frame 1 and a frame 2) 
adjacently to each other on a time axis within image 
information to be encoded, and between these frames, for 
example, an image within a macro block Ml as an image block 
located within the frame 1 shown FIG.l at a time tl is moved 
to a position illustrated as a macro block M2 in FIG.l within 
the frame 2 at a time t2, because of the change of the image 
information from the time tl to the time t2. 

At this time, in the movement compensating process in 
the MPEG method, a common search range W between the 
frame 1 and the frame 2 is firstly established, and then, a 
macro block within the frame 2 having the same image (the 
same pattern) as that within the macro block Ml within the 
frame 1 is searched in the range W. 

In the example shown in FIG.l, the macro block to be 
searched is the macro block M2 on the frame 2. In this case, 
the movement during the time period (from the time tl to the 
time t2) of the image located within the macro block Ml (i.e., 
the movement from the position of the macro block Ml to the 
position of the macro block M2) is represented by a movement 
amount on a two-dimensional coordinate is referred to as a 
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movement vector V. The movement compensating process 
according to the MPEG method does not encode both of the 
frame 1 and the frame 2 by using both of the macro block Ml 
and the macro block M2 as they are. Namely, instead of 
encoding the whole macro block M2, the movement 
compensating process encodes the movement vector V and the 
macro block Ml on the frame 1, as information indicating the 
macro block M2 (in this way, the manner of using the 
movement vector to predict the change of the image in its 
adjacent frame is referred to as an inter-frame prediction). 

By this process, the information amount can be 
extremely reduced to encode the frame 1 and the frame 2, as 
compared with the case in which the respective macro blocks 
are encoded as they are. 

In the present invention, when generating this 
movement vector V, an area of the search range W and the 
search accuracy to generate the movement vector V are 
changed in accordance with the characteristics of the image 
(e.g., the heavy or little movement, the light and shade, or the 
like) within the macro block for each macro block to be 
encoded. Then, the movement vector is generated. 
(II) First Embodiment 

The first embodiment according to the present 
invention including the movement compensating process will 
be described below with reference to FIG. 2 to FIG. 4B. 

As shown in FIG. 2, an image encoding apparatus by 
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means of the MPEG method in the first embodiment is 
provided with an adder 1, a DCT (Discrete Cosine Transform) 
unit 2, a quantizer 3, a reverse quantizer 4, a variable-length 
encoder 5 serving as an encoding device, a reverse DCT unit 6, 
a movement detecting unit 7, a movement compensation 
predicting unit 8 serving as a compensating device and a rate 
controller 9. 

The whole operation will be described below. 

An input signal Sin (digitized for each pixel 
constituting each frame) composed of a plurality of frame 
images inputted to the image encoding apparatus S from the 
external is inputted to the movement detecting unit 7 and also 
inputted to the adder 1. 

Then, the movement detecting unit 7 calculates the 
movement vector V for each frame within the input signal Sin, 
by using a method described later. A corresponding vector 
signal Sv is outputted to the movement compensation 
predicting unit 8. 

On the other hand, a compensation signal Se from the 
movement compensation predicting unit 8 is subtracted from 
the input signal Sin outputted to the adder 1, by the adder 1. 
Then, the subtracted signal is outputted to the DCT unit 2 as 
a subtraction signal Sa. 

Next, the DCT unit 2 performs on the subtraction 
signal Sa the DCT (Discrete Cosine Transform) for 
compressing an amount of information by using the well 
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known technique, and then outputs it to the quantizer 3 as a 
conversion signal Sd* 

Then, the quantizer 3 quantizes the conversion signal 
Sd so as to adapt it to a bit rate indicated by a rate signal Sr 
described later, and then generates a quantization signal Sq, 
and further outputs it to the variable-length encoder 5 and 
the reverse quantizer 4. 

Next, the reverse quantizer 4 performs a reversely 
quantizing process on the quantization signal Sq, and then 
generates a reverse quantization signal Siq, and further 
outputs it to the reverse DCT unit 6. 

Then, the reverse DCT unit 6 performs on the reverse 
quantization signal Siq the reverse DCT (Reverse Discrete 
Cosine Transform) by using the well known technique, and 
then outputs it to the movement compensation predicting unit 
8 as a reverse conversion signal Sid. 

After that, the movement compensation predicting 
unit 8 executes the movement compensating process using the 
inter-frame prediction, in accordance with the reverse 
conversion signal Sid and the movement vector V included in 
the vector signal Sv, and then generates the compensation 
signal Se for compressing the information amount, and 
further outputs it to the adder 1. 

On the other hand, the variable-length encoder 5 
performs a variable-length encoding process on the 
quantization signal Sq, and outputs to the external an output 



27 



signal Sout that is a signal in which the original input signal 
Sin is compressed and encoded by using the MPEG method. 

At this time, the rate controller 9 generates the rate 
signal Sr for optimizing the bit rate in the quantizing 
operation of the quantizer 3, in accordance with the output 
signal Sout, and then outputs the rate signal Sr to the 
quantizer 3. 

The detailed configuration and the operations of the 
movement detecting unit 7 according to the present invention 
will be described below with reference to FIGs.3A, 3B, 4A and 
4B. 

As shown in FIG.3A, the movement detecting unit 7 is 
provided with: a vector generator 10 serving as a generating 
device, a first generating device and a first adding device; a 
vector generator 11 serving as a generating device, a second 
generating device and a second adding device; a comparator 
12 serving as a selecting device; a switch 13 serving as a 
selecting device; and a standardizer 14 serving as a 
standardizing device. 

Operations will be described below. 

At first, for each frame in the input signal Sin, the 
vector generator 10 defines, for example, a horizontal 
direction ± 32 pixels and a vertical direction ± 32 pixels as 
a search range Wl, and then generates a movement vector VI 
at a half-pixel accuracy, for all the pixels within the macro 
block M (that is, executes a so-called entire search), and 
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further outputs a corresponding vector signal Svl to one input 
terminal of the switch 13. 

In parallel to the above-mentioned operation, the 
vector generator 10 generates a range signal Swl 
representing an area of the search range Wl, and then 
outputs the range signal Swl to the standardizer 14. 

On the other hand, for each frame in the input signal 
Sin, the vector generator 11 defines, for example, a horizontal 
direction ± 128 pixels and a vertical direction ± 32 pixels 
as a search range W2, and then generates a movement vector 
V2 at the half-pixel accuracy, for the remaining pixels after a 
predetermined number of pixels which is set in advance are 
thinned out from all the pixels within the macro block M (that 
is, executes a hierarchy search), and then outputs a 
corresponding vector signal Sv2 to the other input terminal of 
the switch 13 and the standardizer 14* 

Then, the standardizer 14 performs a standardizing 
process similar to that of a standardizer 23 described later, on 
each of the inputted vector signal Sv2 and range signal Swl, 
and then outputs the standardized vector signal Sv2 and 
range signal Swl to the comparator 12. 

Accordingly, in accordance with the inputted range 
signal Swl and vector signal Sv2, if the length of the 
generated movement vector V2 is a length exceeding the 
search range Wl (refer to FIG.4B), the comparator 12 
generates a control signal Sc to control the switch 13 so as to 
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select the vector signal Sv2 corresponding to the movement 
vector V2, and then outputs the control signal Sc to the switch 
13. 

On the other hand, if the length of the generated 
movement vector V2 is a length within the search range Wl 
(refer to FIG.4A), the comparator 12 generates the control 
signal Sc to control the switch 13 so as to select the vector 
signal Svl corresponding to the movement vector VI, and then 
outputs the control signal Sc to the switch 13. 

Accordingly, the switch 13 switches the vector signal 
Svl or Sv2 in accordance with the control signal Sc, and 
outputs the vector signal Sv corresponding to the movement 
vector V shown in FIG.l, to the movement compensation 
predicting unit 8. 

The detailed configuration of the vector generator 11 
will be described below with reference to FIG.3B. 

As shown in FIG.3B, the vector generator 11 is 
provided with: a horizontal filter 11a, which limits a 
horizontal frequency band in the input signal Sin in order to 
prevent the extreme expansion of a bit interval of the input 
signal Sin in a horizontal direction after the execution of a 
later-described horizontal thinning out process; a thinning- 
out circuit lib which thins out, every other bit, the bits in the 
horizontal direction in the input signal Sin after the 
limitation of the frequency band to thereby halve the number 
of bits in the horizontal direction, and also separates the 
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signal, after data in the horizontal direction is thinned out, 
into a first field and a second field within one frame, and then 
outputs a signal corresponding to the first signal to a vertical 
filter 11c, and further outputs a signal corresponding to the 
second field to a vertical filter lid; 

the vertical filter 11c which limits a vertical frequency band 
in the signal corresponding to the first field in order to 
prevent the extreme expansion of a bit interval in a vertical 
direction after an execution of a later-described vertically 
thinning-out process is performed on the signal corresponding 
to the first filed; a thinning-out circuit lie which thins out, 
every other bit, bits in the vertical direction in the signal 
corresponding to the first field after the limitation of the 
vertical frequency band to thereby halve the number of bits in 
the vertical direction, and then outputs it to a generator 
circuit llg; the vertical filter lid which limits a vertical 
frequency band in the signal corresponding to the second field 
in order to prevent the extreme expansion of a bit interval in a 
vertical direction after the later-described vertically thinning 
out process is performed on the signal that is outputted by the 
thinning-out circuit lib and corresponds to the second field; a 
thinning-out circuit llf which thins out, every other bit, bits 
in the vertical direction in the signal corresponding to the 
second field after the limitation of the vertical frequency band 
to thereby halve the number of bits in the vertical direction, 
and then outputs it to the generator circuit llg; and the 



31 



generator circuit llg that performs the above-mentioned 
hierarchy search on each field in the input signal Sin in which 
the number of bits in the horizontal direction and the number 
of bits in the vertical direction are respectively thinned out, 
and then generates the vector signal Sv2 corresponding to the 
movement vector V2. 

In addition, if one frame in the input signal Sin is 
composed of, for example, 1920 pixels X 1088 pixels in a hi- 
vision image, one frame is thinned out to 960 pixels X 544 
pixels, in the hierarchy search of the vector generator 11. 
The search range W2 is established under this condition. 
Then, the search of the movement vector is executed. 

As mentioned above, according to the process of the 
movement detecting unit 7 in the first embodiment, the 
movement vector V corresponding to the characteristics of the 
image within the macro block M to be encoded is extracted 
from the movement vectors VI and V2 generated by making 
the search ranges different from each other and also making 
the search accuracies different from each other, with regard 
to the same single macro block M. Thus, it is possible to 
generate the movement vector V generated in accordance with 
the proper search range and search accuracy in response to 
the image within each macro block. 

More concretely, the movement vector VI can be 
generated at a high accuracy, for the macro block M in which 
the movement of the image is minute, and the movement 
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vector V2 can be also generated in a wide search range, for the 
image block in which the movement of the image is large. 

Also, the movement vector generated in accordance 
with the proper search range and search accuracy in response 
to the image within each macro block can be generated 
without excessively increasing the processing amount when 
generating the movement vector V. 

Moreover, the image information can be encoded at a 
higher image quality, as compared with the case in which the 
movement vector is generated at the same uniform search 
range and search accuracy. 
(IE) Second Embodiment 

A second embodiment that is another embodiment of 
the movement detecting unit according to the present 
invention will be described below with reference to FIG. 5. In 
the second embodiment described below, the components of an 
image encoding apparatus except the movement detecting unit 
are same as those of the image encoding apparatus S in the 
first embodiment. Thus, the detailed description thereof are 
omitted. 

In the first embodiment, the area of the search range 
Wl when carrying out the entire search and the length of the 
movement vector V2 obtained by carrying out the hierarchy 
search are compared with each other to thereby generate the 
movement vector V to be detected. However, in the second 
embodiment, a sum of absolute values of differences 
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calculated when generating the movement vector VI and a 
sum of absolute values calculated when generating the 
movement vector V2 are compared with each other to thereby 
generate the movement vector V to be detected. 

That is, as shown in FIG. 5, a movement detecting unit 
50 according to the second embodiment is provided with vector 
generators 20 and 21, a comparator 22, a standardizer 23 and 
the above-mentioned switch 13. 

At this time, similarly to the vector generator 10, for 
each frame in the input signal Sin, the vector generator 20 
firstly defines the horizontal direction ± 32 pixels and the 
vertical direction ± 32 pixels as the search range Wl, and 
then carries out the entire search and thereby generates the 
movement vector VI, and further outputs the corresponding 
vector signal Svl to one input terminal of the switch 13. 

In parallel to the above-mentioned operation, the 
vector generator 20 generates a sum signal Ssl corresponding 
to a sum of absolute values of differences calculated when the 
movement vector VI is generated, that is, a sum of absolute 
values in which absolute values of differences between 
respective one of pixels within the above-mentioned macro 
block M and the corresponding pixel within a frame (in the 
case of FIG.l, the frame 2) targeted by a movement 
compensation are added to each other, with regard to all the 
pixels within the macro block M, and then outputs the sum 
signal Ssl to the standardizer 23. 
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On the other hand, similarly to the vector generator 11, 
for each frame in the input signal Sin, the vector generator 21 
firstly defines the horizontal direction ± 128 pixels and the 
vertical direction ± 32 pixels as the search range W2, and 
then carries out the hierarchy search, and accordingly 
generates the movement vector VI, and further outputs the 
corresponding vector signal Sv2 to the other input terminal of 
the switch 13. 

In parallel to the above-mentioned operation, the 
vector generator 21 generates a sum signal Ss2 corresponding 
to the sum of the absolute values of the differences calculated 
when the movement vector V2 is generated, and then outputs 
the sum signal Ss2 to the standardizer 22. 

Then, for each of the inputted sum signals Ssl and Ss2, 
the standardizer 23 multiplies, for example, the sum of the 
absolute values included in the sum signal Ss2 by 4 times, and 
further outputs the sum of the absolute values included in the 
sum signal Ssl as it is. Alternatively, the standardizer 23 
outputs the sum of the absolute values included in the sum 
signal Ss2 as it is, and further performs a so-called 
standardizing process, such as a process of quartering the 
sum of the absolute values included in the sum signal Ssl and 
other operations, and then outputs the standardized sum 
signals Ssl, Ss2 to the comparator 22. This process of the 
standardizer 23 results from the fact that the sum signal Ssl 
is four times larger than the sum signal Ss2, in the number of 
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pixels corresponding to each of the sum signals. 

Accordingly, if the sum of the absolute values included 
in the standardized sum signal Ssl is equal to or less than the 
sum of the absolute values included in the standardized sum 
signal Ss2, the comparator 22 generates the control signal Sc 
to control the switch 13 so as to select the vector signal Svl 
corresponding to the movement vector VI, and then outputs 
the control signal Sc to the switch 13. 

On the other hand, if the sum of the absolute values 
included in the standardized sum signal Ssl is greater than 
the sum of the absolute values included in the standardized 
sum signal Ss2, the comparator 22 generates the control 
signal Sc to control the switch 13 so as to select the vector 
signal Sv2 corresponding to the movement vector V2, and then 
outputs the control signal Sc to the switch 13. 

Accordingly, the switch 13 switches the vector signal 
Svl or Sv2 in accordance with the control signal Sc, and then 
outputs the vector signal Sv corresponding to the movement 
vector V shown in FIG.l, to the movement compensation 
predicting unit 8. 

According to the above-mentioned operations of the 
movement detecting unit 50 in the second embodiment, the 
movement vector V can be generated so as to keep the above- 
mentioned sum of the absolute values to a minimum in the 
wider search range, namely, so as to indicate the macro block 
M having the closer image. 
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In addition, the movement detecting unit 50 in the 
second embodiment may have a below-described configuration 
other than the above-mentioned configurations. That is, if 
the difference between the sum of the absolute values 
included in the sum signal Ssl and the sum of the absolute 
values included in the sum signal Ss2 is equal to or less than a 
preset predetermined threshold to detect the difference 
between the movement vector VI and the movement vector V2 
at a high accuracy, the comparator 22 generates the control 
signal Sc to control the switch 13 so as to select the vector 
signal Svl corresponding to the movement vector VI, and then 
outputs the control signal Sc to the switch 13. Also, if the 
difference between the sum of the absolute values included in 
the sum signal Ssl and the sum of the absolute values 
included in the sum signal Ss2 is greater than the 
predetermined threshold, the comparator 22 generates the 
control signal Sc to control the switch 13 so as to select the 
vector signal Sv2 corresponding to the movement vector V2, 
and then outputs the control signal Sc to the switch 13. 

Such a configuration enables the movement vector V 
with the high accuracy to be generated even if the difference 
between the above-mentioned sums of the absolute values in 
the movement vector VI and the movement vector V2 is 
minute. 

(IV) Third Embodiment 

A third embodiment that is another embodiment of the 
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movement detecting unit according to the present invention 
will be described below with reference to FIG. 6. In the third 
embodiment described below, the components of an image 
encoding apparatus except the movement detecting unit are 
similar to those of the image encoding apparatus S in the first 
embodiment. Thus, the detailed descriptions thereof are 
omitted. 

In the first embodiment, the area of the search range 
Wl when the entire search is carried out and the length of the 
movement vector V2 obtained by executing the hierarchy 
search are used to generate the movement vector V to be 
detected. However, in the third embodiment, one of the 
movement vectors VI and V2, which is located closer to a 
pre-generated movement vector Vp corresponding to a macro 
block positioned in the periphery of the macro block M from 
which the movement vector V is to be detected is generated as 
the movement vector V to be detected. 

That is, as shown in FIG.6A, a movement detecting 
unit 51 according to the third embodiment is provided with 
vector generators 30 and 31, a comparator 32, a standardizer 
33 and the above-mentioned switch 13. 

At this time, similarly to the vector generator 10 or 20, 
for each frame in the input signal Sin, the vector generator 30 
firstly defines the horizontal direction ± 32 pixels and the 
vertical direction ± 32 pixels as the search range Wl, and 
then carries out the above-mentioned entire search, and 
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thereby generates the movement vector VI, and further 
outputs the corresponding vector signal Svl to one input 
terminal of the switch 13 and the standardizer 33, 

On the other hand, similarly to the vector generator 11 
or 12, for each frame in the input signal Sin, the vector 
generator 31 firstly defines the horizontal direction ± 128 
pixels and the vertical direction ± 32 pixels as the search 
range W2, and then carries out the above-mentioned 
hierarchy search, and accordingly generates the movement 
vector V2, and further outputs the corresponding vector 
signal Sv2 to the other input terminal of the switch 13 and the 
standardizer 33. 

Then, the standardizer 33 performs the standardizing 
process similar to that of the standardizer 23 in the second 
embodiment of FIG. 5, on each of the inputted sum signals Ssl 
and Ss2, and then outputs the standardized sum signals Ssl 
and Ss2 to the comparator 32. 

Accordingly, the comparator 32 compares a pre- 
generated vector signal Svp (which is stored in a memory (not 
shown) when the movement vector Vp corresponding to the 
adjacent or peripheral macro block is generated) representing 
the movement vector Vp corresponding to another macro block 
positioned in the periphery of the macro block M from which 
the movement vector is to be calculated (for example, a macro 
block adjacent or peripheral to the macro block M), with the 
above-mentioned standardized vector signals Svl and Sv2, 
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and then generates the control signal Sc to control the switch 
13 so as to select the vector signal corresponding to the 
movement vector located closer to the movement vector Vp. 

That is, in a case shown in FIG.6B, one of the 
generated movement vectors VI and V2, which is located 
closer to the movement vector Vp, is the movement vector VI. 
Thus, in this case, the comparator 32 generates the control 
signal Sc to control the switch 13 so as to select the vector 
signal Svl corresponding to the movement vector VI. 

Accordingly, the switch 13 selects the moving vector 
closer to the moving vector Vp as the movement vector V from 
which the movement vector is to be detected, in accordance 
with the control signal Sc, and outputs the corresponding 
vector signal Sv out of the vector signals Svl or Sv2 to the 
movement compensation predicting unit 8. 

According to the above-mentioned operations of the 
movement detecting unit 51 in the third embodiment, by using 
the fact that the approximate movement vectors are generated 
between the macro blocks adjacent to each other in many 
cases, it is possible to consider the relative relationship 
between the plurality of macro blocks and thereby generate 
the movement vector V. 
(V) Fourth Embodiment 

A fourth embodiment that is another embodiment of 
the movement detecting unit according to the present 
invention will be described below with reference to FIG. 7. In 
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the fourth embodiment described below, the components of an 
image encoding apparatus except the movement detecting unit 
are similar to those of the image encoding apparatus S in the 
first embodiment. Thus, the detailed descriptions thereof 
are omitted. 

In the above-mentioned first to third embodiments, 
the movement detecting unit has two vector generators for 
generating the movement vector. However, other than that, 
the movement detecting unit may have three or more vector 
generators which are different from each other in the search 
range and the search accuracy, as the vector generator, and 
further utilize the combination of the above-mentioned 
methods in the first to third embodiments, as the method of 
generating the final movement vector V. 

That is, for example, in the fourth embodiment shown 
in FIG. 7, a movement detecting unit 52 is provided with: three 
vector generators 40 to 42 which are different from each other 
in the search range and the search accuracy and generate a 
vector signal Svl corresponding to a movement vector VI, a 
vector signal Sv2 corresponding to a movement vector V2 and 
a vector signal Sv3 corresponding to a movement vector V3, 
respectively; comparators 43 and 45, switches 44 and 46, and 
standardizers 47 and 48. 

As for the search range and the search accuracy of 
each of the vector generators 40 to 42, the vector generator 40 
has the narrowest search range, and the vector generator 42 
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has the widest search range. On the other hand, the vector 
generator 40 has the highest search accuracy, and the vector 
generator 42 has the lowest search accuracy. 

Then, the vector generator 40 among the three vector 
generators 40 to 42 outputs the vector signal Svl to one input 
terminal of the switch 46 and also outputs it to the 
standardizer 48. 

In parallel to the above-mentioned operation, the 
vector generator 40 generates a range signal Swl indicating 
the area of the search range Wl thereof, and then outputs the 
range signal Swl to the comparator 45. 

Also, the vector generator 41 generates the vector 
signal Sv2, and further generates a sum signal Ss2 
corresponding to the sum of the absolute values of the above- 
mentioned differences calculated when the movement vector 
V2 is generated, and then outputs the sum signal Ss2 to the 
standardizer 47. 

Moreover, the vector generator 42 generates the vector 
signal Sv3, and further generates a sum signal Ss3 
corresponding to the sum of the absolute values of the above- 
mentioned differences calculated when the movement vector 
V3 is generated, and then outputs the sum signal Ss3 to the 
standardizer 47. 

Then, the standardizer 47 standardizes the sum 
signals Ss2 and Ss3, respectively, as mentioned above, and 
then outputs them to the comparator 43. 
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Accordingly, if the sum of the absolute values included 
in the standardized sum signal Ss2 is equal to or less than the 
sum of the absolute values included in the standardized sum 
signal Ss3, the comparator 43 generates a control signal Scl 
to control the switch 44 so as to select the vector signal Sv2 
corresponding to the movement vector V2, and then outputs 
the control signal Scl to the switch 44. 

On the other hand, if the sum of the absolute values 
included in the standardized sum signal Ss2 is greater than 
the sum of the absolute values included in the standardized 
sum signal Ss3, the comparator 43 generates the control 
signal Scl to control the switch 44 so as to select the vector 
signal Sv3 corresponding to the movement vector V3, and then 
outputs the control signal Scl to the switch 44. 

Accordingly, the switch 44 switches the vector signal 
Sv2 or Sv3 in accordance with the control signal Scl, and then 
outputs a selection signal Ssw to the other input terminal of 
the switch 46, and also outputs it to the standardizer 48. 

Then, the standardizer 48 standardizes the vector 
signal Svl and the selection signal Ssw, respectively, as 
mentioned above, and outputs them to the comparator 45. 

By this, in accordance with the inputted range signal 
Swl, if the length of the movement vector included in the 
standardized selection signal Ssw is a length beyond the 
search range Wl, the comparator 45 generates a control signal 
Sc2 to control the switch 46 so as to select the selection signal 
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Ssw corresponding to the pertinent movement vector, and 
then outputs the control signal Sc2 to the switch 46. 

On the other hand, if the length of the movement 
vector included in the standardized selection signal Ssw is a 
length belonging to the search range Wl, the comparator 45 
generates the control signal Sc2 to control the switch 46 so as 
to select the vector signal Svl, and then outputs the control 
signal Sc2 to the switch 46. 

Accordingly, the switch 46 switches the vector signal 
Svl or the selection signal Ssw in accordance with the control 
signal Sc2, and then outputs the vector signal Sv to the 
movement compensation predicting unit 8. 

According to the above-mentioned operations of the 
movement detecting unit 52 in the fourth embodiment, it is 
possible to achieve the effect of the combination of the first 
embodiment and the second embodiment. 
(VI) Modified Embodiment 

The modified embodiment of the present invention will 
be described below with reference to FIG. 8. 

In the above-mentioned respective embodiments, each 
movement detecting unit generates the movement vector V by 
using only the input signal Sin. However, other than it, the 
movement vector may be generated by also using an image 
which is again prepared by the above-mentioned encoding 
process, by the movement detecting unit. 

That is, as shown in FIG. 8, instead of the reverse DCT 
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unit 6 and the movement detecting unit 7 of the image 
encoding apparatus S in the first embodiment, an image 
encoding apparatus S ? in the modified embodiment may be 
provided with: a reverse DCT unit 6 ? which generates a re- 
composed signal Sdd including the re-composed image 
together with the reverse DCT signal Sid based on the reverse 
quantization signal Siq; and a movement detecting unit T 
which generates a movement vector by using any one of the 
methods according to the first to fourth embodiments in 
accordance with the input signal Sin and the re-composed 
signal Sd, and accordingly generates a vector signal Sv, in 
addition to the adder 1, the DCT unit 2, the quantizer 3, the 
reverse quantizer 4, the variable-length encoder 5, the 
movement compensation predicting unit 8, and the rate 
controller 9. 

The generation of the movement vector in accordance 
with this configuration can provide the effect of reducing the 
error of the prediction, in addition to the above-mentioned 
effects of the respective embodiments. 

In addition, the case of using the single macro block as 
the pixel block has been described in the above-mentioned 
respective embodiments and modified embodiment. However, 
other than that case, a plurality of macro blocks adjacently to 
each other may be collectedly utilized as the single pixel 
block. 

(VII) Example 
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An example of an image which is encoded by using the 
first embodiment among the above-mentioned respective 
embodiments will be described with reference to the following 
tables. 

In this example, the 20 th image (hereafter, the 20th 
standard image is referred to as "soccer") and the 35 th image 
(hereafter, the 35th standard image is referred to as "horse 
race") are used from among the standard images for a hi- 
vision television determined by an image information media 
society (ITE). The simulation is performed by using the 
configuration of the movement detecting unit in the first 
embodiment. 

That is, the average of S/N ratios of respective 
pictures (an I (Intra-coded) picture, a P (Predictive) picture 
and a B (Bidirectionally) picture) included in the encoded 
output signal Sout with respect to the images within the 
original input signal Sin are measured as for the case of 
generating all the movement vectors by using the entire 
search (in the following tables, referred to as "Entire Search"), 
the case of generating all the movement vectors by using the 
hierarchy search (in the following tables, referred to as 
"Hierarchy Search") and the case of generating the movement 
vectors by using the method of the first embodiment (in the 
following tables, referred to as "Adaptation Switch"), when 
compressing the data to the bit rates as compression targets 
(15 Mbps (bit per second), 20 Mbps and 30 Mbps) respectively 
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shown in TABLE (a) in FIG. 9 and TABLE (b) in FIG. 10. In 
addition, "Total" in each column of these tables indicates the 
average of the S/N ratio of respective one of the pictures 
within the pertinent column. 

Now explaining the respective standard images used 
in this experiment, the above-mentioned "soccer" photographs 
a certain soccer game, and this is the image in which a field 
for the game is widely illustrated. That is, the lawn within a 
stadium occupies the majority of a screen, a line of a field 
track is seen on the other side, and audience seats are 
positioned beyond it. Then, although soccer players wear 
brightly-colored uniforms, an individual player is tinily 
illustrated. Moreover, a camera is laterally shifting in 
coincidence with a movement of a ball. Furthermore, the 
whole screen is laterally shifted in conjunction with the 
lateral shift. 

On the other hand, the above-mentioned "horse race" 
is a scene in which a plurality of horses are running in a 
racetrack. It is the horses that are actually running. 
However, a camera chasing the horses is also shifting. 
Accordingly, this is the image in which a background is 
shifted at a sufficient speed. 

The TABLE (a) shows the result of the experiment 
with regard to the image "soccer", and the TABLE (b) shows 
the result of the experiment with regard to the image "horse 
race". 
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From the viewpoint of these experiment results, it is 
understood that the utilization of the method in the first 
embodiment improves the S/N, as compared with the case of 
executing the entire search or the hierarchy search for all the 
macro blocks, in any case of the compression targets. 

By these results of the experiment, it can be confirmed 
that the present embodiment is especially effective for the 
image in which the movement is rather active. 

The invention may be embodied in other specific forms 
without departing from the spirit or essential characteristics 
thereof. The present embodiments are therefore to be 
considered in all respects as illustrative and not restrictive, 
the scope of the invention being indicated by the appended 
claims rather than by the foregoing description and all 
changes which come within the meaning and range of 
equivalency of the claims are therefore intended to be 
embraced therein. 

The entire disclosure of Japanese Patent Application 
No. 10-92673 filed on March 20, 1998 including the 
specification, claims, drawings and summary is incorporated 
herein by reference in its entirety. 
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WHAT IS CLAIMED IS: 



1. A movement vector generating apparatus for 
generating a movement vector for a movement compensation 
by means of an inter-frame prediction, when encoding a preset 
image information including an image of a plurality of frames 
by using the movement compensation, said apparatus 
comprising: 

a plurality of generating devices each for generating 
the movement vector corresponding to a search range and a 
search accuracy between one frame and another frame, for 
each pixel block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, said generating devices respectively using search 
ranges different from each other and search accuracies 
different from each other; and 

a selecting device for selecting one of movement 
vectors generated by said generating devices, in accordance 
with characteristics of the image in said each pixel block, and 
then outputting the selected movement vector corresponding 
to said each pixel block. 

2. A movement vector generating apparatus according to 
Claim 1, wherein said generating devices comprise: 

a first generator for generating a first movement 
vector, with a preset first range as the search range; and 
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a second generator for generating a second movement 
vector at the search accuracy lower than that of the first 
movement vector, with a preset second range wider than the 
first range as the search range. 

3. A movement vector generating apparatus according to 
Claim 2, wherein said selecting device outputs the second 
movement vector as the selected movement vector if a length 
of the second movement vector is a length beyond the search 
range in the first generating device, and outputs the first 
movement vector as the selected movement vector if the 
length of the second movement vector is a length belonging to 
the search range in the first generating device. 

4. A movement vector generating apparatus according to 
Claim 2, wherein said selecting device comprises: 

a first adding device for adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said first generating device, to 
generate a first absolute value sum; 

a second adding device for adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
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pixels in the pixel block, in said second generating device, to 
generate a second absolute value sum; and 

a standardizing device for standardizing the 
generated first and second absolute value sums, respectively, 

said selecting device comparing the standardized first 
absolute value sum with the standardized second absolute 
value sum, and outputting the second movement vector as the 
selected movement vector if the standardized first absolute 
value sum is greater than the standardized second absolute 
value sum. 

5. A movement vector generating apparatus according to 

Claim 2, wherein said selecting device comprises: 

a first adding device for adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said first generating device, to 
generate a first absolute value sum; 

a second adding device for adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said second generating device, to 
generate a second absolute value sum; and 

a standardizing device for standardizing the 



generated first and second absolute value sums, respectively, 
said selecting device comparing the standardized first 
absolute value sum with the standardized second absolute 
value sum, outputting the first movement vector as the 
selected movement vector if a difference between the 
standardized first absolute value sum and the standardized 
second absolute value sum is not greater than a 
predetermined threshold which is set in advance to detect a 
difference between the first movement vector and the second 
movement vector at a high accuracy, and outputting the 
second movement vector as the selected movement vector if 
the difference between the standardized first absolute value 
sum and the standardized second absolute value sum is 
greater than the predetermined threshold. 

6. A movement vector generating apparatus according to 
Claim 2, wherein said selecting device outputs one of the first 
and second movement vectors which is closer to the selected 
movement vector corresponding to another pixel block located 
adjacent to one pixel block as the selected movement vector 
corresponding to said one pixel block from which the first 
movement vector and the second movement vector are 
generated. 

7. An image encoding apparatus comprising 

(a) a movement vector generating apparatus for 
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generating a movement vector for a movement compensation 
by means of an inter-frame prediction, when encoding a preset 
image information including an image of a plurality of frames 
by using the movement compensation, said apparatus 
comprising: 

a plurality of generating devices each for generating 
the movement vector corresponding to a search range and a 
search accuracy between one frame and another frame, for 
each pixel block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, said generating devices respectively using search 
ranges different from each other and search accuracies 
different from each other; and 

a selecting device for selecting one of movement 
vectors generated by said generating devices, in accordance 
with characteristics of the image in said each pixel block, and 
then outputting the selected movement vector corresponding 
to said each pixel block, 

(b) a compensating device for performing the movement 
compensation on the basis of the selected movement vector 
outputted from the selecting device, to output a compensation 
signal, and 

(c) an encoding device for encoding the image information 
on the basis of the compensation signal. 

8. A movement vector generating method of generating a 
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movement vector for a movement compensation by means of an 
inter-frame prediction, when encoding a preset image 
information including an image of a plurality of frames by 
using the movement compensation, said method comprising: 

a plurality of generating processes each of generating 
the movement vector corresponding to a search range and a 
search accuracy between one frame and another frame, for 
each pixel block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, said generating processes respectively using search 
ranges different from each other and search accuracies 
different from each other; and 

a selecting process of selecting one of movement 
vectors generated by the generating processes, in accordance 
with characteristics of the image in said each pixel block, and 
then outputting the selected movement vector corresponding 
to said each pixel block. 

9. A movement vector generating method according to 

Claim 8, wherein said generating processes comprise: 

a first generating process of generating a first 
movement vector, with a preset first range as the search 
range; and 

a second generating process of generating a second 
movement vector at the search accuracy lower than that of the 
first movement vector, with a preset second range wider than 
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the first range as the search range. 

10. A movement vector generating method according to 
Claim 9, wherein said selecting process outputs the second 
movement vector as the selected movement vector if a length 
of the second movement vector is a length beyond the search 
range in the first generating process, and outputs the first 
movement vector as the selected movement vector if the 
length of the second movement vector is a length belonging to 
the search range in the first generating process. 

11. A movement vector generating method according to 
Claim 9, wherein said selecting process comprises: 

a first adding process of adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said first generating device, to 
generate a first absolute value sum; 

a second adding process of adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said second generating device, to 
generate a second absolute value sum; and 

a standardizing process of standardizing the 
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generated first and second absolute value sums, respectively, 
said selecting process comparing the standardized 
first absolute value sum with the standardized second 
absolute value sum, and outputting the second movement 
vector as the selected movement vector if the standardized 
first absolute value sum is greater than the standardized 
second absolute value sum. 



12. A movement vector generating method according to 
Claim 9, wherein said selecting process comprises: 

a first adding process of adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said first generating device, to 
generate a first absolute value sum; 

a second adding process of adding together absolute 
values of differences between respective one of the pixels in 
the pixel block and its corresponding pixel in the frame 
targeted by the movement compensation, as for all of the 
pixels in the pixel block, in said second generating device, to 
generate a second absolute value sum; and 

a standardizing process of standardizing the 
generated first and second absolute value sums, respectively, 

said selecting process comparing the standardized 
first absolute value sum with the standardized second 
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absolute value sum, outputting the first movement vector as 
the selected movement vector if a difference between the 
standardized first absolute value sum and the standardized 
second absolute value sum is not greater than a 
predetermined threshold which is set in advance to detect a 
difference between the first movement vector and the second 
movement vector at a high accuracy, and outputting the 
second movement vector as the selected movement vector if 
the difference between the standardized first absolute value 
sum and the standardized second absolute value sum is 
greater than the predetermined threshold. 

13. A movement vector generating method according to 
Claim 9, wherein said selecting process outputs one of the 
first and second movement vectors which is closer to the 
selected movement vector corresponding to another pixel 
block located adjacent to one pixel block as the selected 
movement vector corresponding to said one pixel block from 
which the first movement vector and the second movement 
vector are generated. 

14. An image encoding method comprising 

(a) a movement vector generating method of generating a 
movement vector for a movement compensation by means of an 
inter-frame prediction, when encoding a preset image 
information including an image of a plurality of frames by 
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using the movement compensation, said method comprising: 

a plurality of generating processes each of generating 
the movement vector corresponding to a search range and a 
search accuracy between one frame and another frame, for 
each pixel block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, said generating processes respectively using search 
ranges different from each other and search accuracies 
different from each other; and 

a selecting process of selecting one of movement 
vectors generated by the generating processes, in accordance 
with characteristics of the image in said each pixel block, and 
then outputting the selected movement vector corresponding 
to said each pixel block, 

(b) a compensating process of performing the movement 
compensation on the basis of the selected movement vector 
outputted from the selecting process, to output a 
compensation signal, and 

(c) an encoding process of encoding the image information 
on the basis of the compensation signal. 
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ABSTRACT OF THE DISCLOSURE 



A movement vector generating apparatus generates a 
movement vector for a movement compensation by means of an 
inter-frame prediction, when encoding a preset image 
information including an image of a plurality of frames by 
using the movement compensation. The movement vector 
generating apparatus is provided with a plurality of 
generating devices each for generating the movement vector 
corresponding to a search range and a search accuracy 
between one frame and another frame, for each pixel block e.g., 
each macro block which is located within said one frame 
respectively in the image information and includes a plurality 
of pixels, the generating devices respectively using search 
ranges different from each other and search accuracies 
different from each other. The movement vector generating 
apparatus is also provided with a selecting device for 
selecting one of movement vectors generated by the 
generating devices, in accordance with characteristics of the 
image in said each pixel block, and then outputting the 
selected movement vector corresponding to said each pixel 
block. 
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FIG. 9 

socce r 



b i t-rate 
[Mbps] 


ENTIRE 
SEARCH [dB] 


HIERARCHY 
SEARCH [dB] 


ADAPTAT I ON 
SWITCH [dB] 


15 


I : 29. 995 
P: 27. 362 
B: 27. 860 
Total: 27. 832 
(-0. 304) 


1 : 30. 559 
P: 27. 728 
B: 28. 023 
Total: 28. 070 
(-0. 066) 


I : 30. 422 
P: 27. 725 
B: 28. 131 
Total: 28. 136 
(best) 


20 


1:31. 340 
P: 28. 285 
B: 28. 379 
Total: 28. 498 
(-0. 235) 


1 : 31 . 635 
P: 28. 510 
B: 28. 496 
Total: 28. 552 
(-0. 081 ) 


1:31.618 
P: 28. 525 
B: 28. 608 
Total: 28. 733 
(best) 


30 


1 : 32. 982 
P: 29. 605 
B: 29. 096 
Total: 29.406 
(-0. 164) 


1 : 33. 121 
P: 29. 726 
B: 29. 164 
Total: 29. 490 
(-0. 080) 


I : 33. 140 
P: 29. 749 
B: 29. 267 
Total: 29. 570 
(best) 



TABLE (a) 



FIG. 10 

horse race 



b i t-rate 
[Mbps] 


ENTIRE 
SEARCH [dB] 


H 1 ERARCHY 
SEARCH [dB] 


ADAPTAT 1 ON 
SWITCH [dB] 


15 


1 : 34. 473 
P: 31 . 805 
B: 32. 014 
Total: 32. 083 
(-0. 491 ) 


1 : 35. 094 
P: 32. 330 
B: 32.465 
Total: 32. 561 
(-0. 013) 


1 : 35. 054 
P: 32. 278 
B: 32. 506 
Total: 32. 574 
(best) 


20 


I : 36. 017 
P: 32. 857 
B: 32. 593 
Total: 32. 825 
(-0. 319) 


1 : 36. 340 
P: 33. 1 13 
B: 32. 892 
Total: 33. 1 14 
(-0. 030) 


1 : 36. 342 
P: 33. 098 
B: 32. 940 
Total: 33. 144 
(best) 


30 


I : 37. 582 
P: 34. 143 
B: 33. 251 
Total: 33. 668 
(-0. 166) 


I : 37. 778 
P: 34. 245 
B: 33. 383 
Total: 33. 795 
(-0. 039) 


1 : 37. 801 
P: 34. 246 
B: 33.435 
Total: 33. 834 
(best) 



TABLE (b) 



RULE 63 (37 C.F.R. 1.63) Nixnn * v^«h y e pc (5/93) 

DECLARATION AND POWER OF ATTORNEY 
FOR PATENT APPLICATION 
IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

As a below named inventor, I hereby declare that my residence, post office address and citizenship arc as stated below next to my name, and I 
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GENERATING APPARATUS AND METHOD, AND IMAGE ENCODING APPARATUS AND METHOD 
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[ ] was filed on as U.S. Application Serial No. 
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and (if applicable to U.S. or PCT application) was amended on 
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sjbelieved to be true; and further that these statements were made with the knowledge that willful false statements and the like so made are 
y^inishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false statements may 
Jliopardize the validity of the application or any patent issued thereon. 

M^nd I hereby appoint NIXON & VANDERHYE P.C., 1100 North Glebe Road, 8th Floor, Arlington, Virginia 22201-4714, telephone 
Hfkjmber (703) 816-4000 (to whom all communications are to be directed), and the following attorneys thereof (of the same address) individually 
ffhd collectively my attorneys to prosecute this application and to transact all business in the Patent and Trademark Office connected therewith and 
jiith' the resulting patent: Arthur R. Crawford, 25327; Larry S. Nixon, 25640; Robert A. Vanderhye, 27076; James T. Hosmer, 30184; Robert W. 
'Saris, 31352; Richard G. Besha, 22770; Mark E. Nusbaum, 32348; Michael J. Keenan, 32106; Bryan H. Davidson, 30251; Stanley C Spooner, 
'•&393; Leonard C. Mitchard, 29009; Duane M. Byers, 33363; Paul J. Henon, 33626; Jeffry H. Nelson, 30481; John R. Lastova, 33149; H. Warren 
Burnam.Jr., 29366; Thomas E. Byrne, 32205; Mary J. Wilson, 32955. 
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